FishSizer: Software solution for efficiently measuring larval fish size

Abstract Length and depth of fish larvae are part of the fundamental measurements in many marine ecology studies involving early fish life history. Until now, obtaining these measurements has required intensive manual labor and the risk of inter‐ and intra‐observer variability. We developed an open‐source software solution to semi‐automate the measurement process and thereby reduce both time consumption and technical variability. Using contrast‐based edge detection, the software segments images of a fish larva into “larva” and “background.” Length and depth are extracted from the “larva” segmentation while taking curvature of the larva into consideration. The graphical user interface optimizes workflow and ease of usage, thereby reducing time consumption for both training and analysis. The software allows for visual verification of all measurements. A comparison of measurement methods on a set of larva images showed that this software reduces measurement time by 66%–78% relative to commonly used software. Using this software instead of the commonly used manual approach has the potential to save researchers from many hours of monotonous work. No adjustment was necessary for 89% of the images regarding length (70% for depth). Hence, the only workload on most images was the visual inspection. As the visual inspection and manual dimension extraction works in the same way as currently used software, we expect no loss in accuracy.


| INTRODUC TI ON
Larval growth rate is a fundamentally important life history trait directly linked to fish population productivity and persistence. For many fish populations, fast-growing larvae have higher survival rates, largely by reducing larval stage duration in which they are exposed to higher predation risk (Houde, 2008). Also, larger body sizes are associated with increased feeding and swimming capacity of larvae, allowing them to search larger water volumes, increase encounter rates with prey, feed on larger organisms, and have better escape responses from predators (Hale, 1996;Miller et al., 1988;Munk & Kiorboe, 1985;Peck et al., 2012). Therefore, body length is an essential measurement in studies of larval fish ecology (Bils et al., 2017), development (Fuiman et al., 1998), physiology Petereit et al., 2008), evolution (Oomen & Hutchings, 2015Oomen et al., 2021), reproductive biology (Roney et al., 2018), and aquaculture (Blanco et al., 2017). Besides length, body depth is also valuable, as it can be used as a proxy for muscle development (i.e., myomere height) and potential starvation resistance Peña et al., 2021).
Body size measurements of larval fish are generally taken on individuals using a camera attached to a stereomicroscope. On the generated images, body length and depth are measured manually using a dedicated software, either open-source (e.g., ImageJ) or proprietary (e.g., Leica Application Suite). Body length is generally estimated as notochord length (tip of the snout to the end of the notochord) during the preflexion stage and as standard length (tip of the snout to the posterior extremity of the hypural plate) afterwards (Kahn et al., 2004). Body depth is generally measured as maximum depth at the head, at the anus, or at the caudal peduncle. Even with the aid of dedicated software, these manual measurements are time consuming because a typical laboratory study can generate thousands of images (e.g., 4489 by Roney et al. (2018)). This manual work also has high potential for introducing intra-and inter-observer variability, potentially leading to increased measurement errors.
Here, we introduce a novel, open-source application, FishSizer, to semi-automate measurements of larval fish length and myomere depth. FishSizer addresses the two major problems of the manual method used to estimate body size in larval fish by considerably reducing the amount of manual work and potentially decreasing technical variability. Although this application was developed specifically for use with fish larvae, it should be useful for older life stages of fish (and possibly other animals) for which satisfactory images are available.

| Overall method
All coding was done in Matlab 2020a (The MathWorks Inc.) and compiled into a single file for installation on computers without Matlab. Determination of length and depth of fish larvae from images is based on A sequence of two procedures: (1) producing a mask (a binary image the same size as the original image) segmented into "larva" and "background" and (2) determining length and depth based on this "larva" segmentation.

| Segmentation
For morphometrics assessments, pictures of anesthetized individual larvae laying on a microscope slide are taken under a stereomicroscope. During a single session, a user may typically take hundreds of pictures of individual larvae, so the zoom level is typically fixed in order to reduce the manipulation time. This procedure results in the background being mostly uniform (i.e., low in contrast) with the exception of lines arising from scratches on the glass slide and water drops. Due to this low-contrast background, segmentation is based on edge detection. Edge detection is a method of establishing regions where the contrast between neighboring pixels in an image is above a certain preset threshold. Edge detection has the advantage of being applicable across species and/or stage of the larvae being measured. More advanced methods, such as deep learning, will in most cases need retraining of the network to correctly segment species not previously encountered by the network (Kvaestad et al., 2022). Deep learning has the additional drawback of having higher hardware demands compared to this less computationally intensive approach (LeCun, 2019). In order to determine a robust edge detection threshold across a wide range of images, we determine the maximum contrast present in the image and set the threshold as a customizable fraction of this value. Many images used for larval fish length measurements contain scale bars or other high-contrast objects. To avoid basing the contrast threshold on these artifacts, this software offers the option of establishing a region of interest (ROI) within which the segmentation is contained.
Running the edge detection algorithm on an image creates a binary mask with pixel values of 1 at edges and 0 at other locations (see Figure 1 for the segmentation process illustrated). The aim is to have a complete outline of the larva and subsequentially fill in this outline. In practice, the detected outline can be incomplete, leaving small gaps and resulting in faulty segmentation. We therefore dilate . For further analysis, we extract orientation for an ellipse that best represents this region.
In order to minimize the need to adjust parameter settings, and make the software as user friendly as possible, we aimed to make the segmentation as robust as possible. We found that a Gaussian image filter with a standard deviation of four pixels followed by an image sharpening algorithm with the same standard deviation significantly improved segmentation (e.g., Figure 3). When applying a Gaussian image filter, the value of each pixel is influenced by the value of all neighboring pixels. Hence, small regions in the outline with low contrast can have increased contrast after the procedure and, therefore, a better chance of complete edge detection.

| Length and depth measurement
Length and depth estimation is done using the segmentation mask.
The first step is to rotate the mask using the orientation of the major axis of the ellipse fitted to the larva outline. This rotation is done using the imrotate.m function in MATLAB. By default, imrotate uses nearest-neighbor interpolation, setting the values of pixels in the final image that are outside the rotated image to 0. Alignment with the horizontal (X) axis allows the use of a polynomial regression for length estimation. It also allows depth estimation based solely on the Y component at a customizable location relative to the length (X axis). The next step is to establish which end is the head end by dividing the larva into two halves of equal length and determining which end contains more pixels in the segmented mask. This procedure assumes that the anterior half of the larva is larger than the posterior half.
Length estimation is based on a polynomial regression line through the larva segmentation that best describes the curvature of the larva. As larval fish are often curved, we use second-order or greater polynomial regression. Presence of a large yolk sac affects the regression, such that a larva with straight notochord and large yolk sac will yield a curved regression that does not accurately represent the larva's length. To avoid this, we base the order of the regression on the curvature of the tail alone. The greater the curvature, the higher the order of the polynomial regression. A lower-order regression on a larva predicted to be straight will result in a straighter regression and hence a more precise length estimation. Therefore, a second-order polynomial regression is computed for the tail part of the larva and, since the larva is rotated to be horizontal, both coefficients of this regression indicate the amount of curvature of the entire larva. If the absolute value of the second-degree coefficient is >0.1 or the absolute value of the first-degree coefficient is >0.5, a third-order polynomial regression is used for the entire larva. Otherwise, a second-order polynomial regression is used.
Depth is determined at a customizable location relative to the length of the larva (X axis in Figure 2, window 2). Because the segmented larvae are rotated to be horizontal, a good approximation of the depth is the difference between the maximum and minimum Y values at the user-defined X location. To make depth determination less dependent on precise point-to-point segmentation, depth is not calculated at a single point but rather as the median of all depths calculated at the user-defined location ±5% of the length of the larva.
For visual verification, length and depth estimation lines are shown in the main graphical user interface (Figure 2 window 2).

| Main GUI window description
The interface of this software is split into two main graphical user interfaces: the main GUI and the loading GUI. The main graphical user interface contains four windows and five button panels as seen in Figure

| Main GUI button description
Buttons in the GUI are grouped according to function (Figure 2). the user to manually draw lines in top left window to measure length and depth. To the right of these two buttons is a panel for setting the number of line segments needed to make the measurement. The default for measuring length is 2, as most larvae are slightly curved and hence a line consisting of two segments will give a more accurate estimate of the length than one straight line. If the number of segments is set greater than needed for a particular larva, pressing "enter" after marking fewer segments will complete the measurement. The default number of segments for manually measuring depth is 1, which usually is sufficient. To the right are buttons for cancelling the manual length or depth estimations and reverting to the automated estimations.
Below the top right window are controls for navigating the image files and exporting data to a comma-delimited file (.csv). To the left is a slider for selecting images from the loaded dataset. For optimized workflow, hotkey functions are associated with this. Pressing "a" selects the image prior to the active image, pressing "s" selects the image after the active image. To the right is the export data button. Data can be exported both with and without calibration into millimeters (mm). The "export data" button is yellow if data have not been calibrated (measurement reported in units of pixels) and green if calibration into millimeters (mm) has been applied to the dataset.

| Loading GUI description
This GUI provides the options to (1) load a test image for testing and adjusting the segmentation settings, and (2)  sets the fraction of maximum contrast (see section 2.2). "Dilation" sets the number of dilations applied to each edge-detected pixel (see section 2.2)." Depth offset" determines the location of the depth measurement in percentage of total larval length from the head. If set to 50, depth is measured at the midpoint of the larva along the X axis in window 2 of the main GUI.
The ROI panel contains three buttons for: (1) setting a rectangular ROI, (2) setting a circular ROI, and (3)

| Workflow
A typical workflow with a set of images is first to confirm or change the segmentation settings on the loading GUI. It is recommended to load a test image and try settings before loading the entire data- file or by creating a new one (see manual for details). After calibration, the "Load dataset" button is green to reflect that calibration has been done. After visual validation of correct segmentation settings, images are loaded with the "Load images" button, after which a progress bar appears. When the progress bar disappears, the slider below window 2 in the main GUI, as well as hotkeys "a" and "s", can be used to navigate the dataset. If automatic estimation is satisfactory, no additional work is needed, and the user can move to the next image. If the automatic estimation is unsatisfactory, manual length and depth extraction is done in the same way as ImageJ. The "Export data" button is green if calibration has been performed and pressing the button will produce a comma-delimited file with five (without calibration) or nine (with calibration) columns containing data as described in Table 1. Time savings were calculated compared to ImageJ (Schneider et al., 2012), a commonly used software for this task. Two independent observers went through the test dataset and extracted length and depth in FishSizer, including visually verifying all measurements.

| PERFORMAN CE E VALUATION
Compared to extracting the same measurements from the same dataset in ImageJ, a time saving of 66 and 78% was observed for observer 1 (R1) and 2 (R2), respectively. Intra-observer variability is expected to be lower using FishSizer compared to ImageJ as the threshold for accepting automatic parameter extraction will remain fairly constant for each observer. For manually extracted parameters in FishSizer, variability is expected to be the same as ImageJ, as the process for manual extraction is the same in both software packages.
To examine inter-observer variability compared to ImageJ, we analyzed data from two observers to compare percentage-wise differences in length and depth measurements across 101 haphazardly chosen tuna larva images ( Figure 6). For length we found a deviation between the two observers of 2.69%±3.35% (mean±std) for FishSizer compared to 3.13% ± 4.54% for ImageJ. For depth we found a mean of 15.6% ± 14.6% for FishSizer and 19.8% ± 34.0% for ImageJ. Therefore, there were a small but not significant difference between the two software packages for both parameters.
For visual representation of the variability using a Bland-Altman graph, see Figure 6. One important aspect of variability is human error. In FishSizer, all handling of parameters and data output is automatic and linked to the image file name. Therefore, FishSizer has less potential to introduce human errors than other programs which can require manual data curation (e.g., using copy/paste), such as ImageJ.
F I G U R E 5 Length (a) and depth (b)

| CON CLUS I ON AND FUTURE DIREC TIONS
The partial automation of obtaining larval morphological measurements in this software saves a considerable amount of time compared to the manual procedure used so far. Even when all measurements obtained via our software were manually verified and corrected, we experienced a time saving of 66%-78%.
When tested across a range of species, we found the software to achieve good segmentation for a wide range of species like Atlantic cod, Atlantic bluefin tuna, Pufferfish, European plaice, and Southern flounder using a rectangular ROI, and Atlantic herring using a circular ROI. (Figure 7). As the parameter extraction relies heavily on segmentation, we expect that the accuracy for these and similar species will be the same as seen for the tuna statisti- contrast detection, we found the quality of images to be essential.
High-contrast artefacts in direct contact with the larva can hinder correct segmentation and thereby correct parameter extraction ( Figure 8).
A logical future direction for this software will be to include deep learning. This current version of FishSizer can help create large datasets of segmented larva images to be used as ground truth for training neural networks. Deep learning will not only facilitate analysis of images with more than one larva, a sought after feature for this kind of software, but it will also open the door to automatic species and/or stage identification of larval fish, which is far behind other pelagic marine organisms such as plankton (Guo et al., 2021).

ACK N OWLED G M ENTS
We appreciate the help of Fredrick Nicolai Clausen and Paul Elias Homme during the initial development of the user interface, Björn Illing for feedback that greatly improved the user experience, and Griet Nobis for early work on the topic that helped to inspire this project.

CO N FLI C T O F I NTE R E S T
All authors declare no conflict of interest.

DATA AVA I L A B I L I T Y S TAT E M E N T
Compiled installable software, manual, and MATLAB files used for compiling are available at: https://doi.org/10.5281/zenodo.5833209 (https://github.com/jeppe have/FishS izer).

O RCI D
F I G U R E 8 Example of faulty segmentation of an Atlantic bluefin tuna larva. In this case, the strong contrast of the water edge to the left is connected to the contrast associated with the larva itself. As all areas directly connected to the borders of the image are ignored, a small area to the right is chosen as largest area surrounded by a complete outline